Fail when grype cant check for db update #1247

shanedell · 2023-04-20T18:14:00Z

Fail when grype cant check for db update

Closes #310

Before change when running:

docker run -v $PWD:/grype -w /grype -d --name grype-test --rm -it golang:1.19 tail -f /dev/null
docker exec -w /grype -it grype-test bash -c "go install"
docker exec -it grype-test bash -c 'echo "nameserver 8.8.8.9" > /etc/resolv.conf'
docker exec -w /grype -it grype-test bash -c "go run main.go db update"

It would output:

 ✔ Vulnerability DB        [no update available]
No vulnerability database update available

and exit with 0.

After update when running:

docker run -v $PWD:/grype -w /grype -d --name grype-test --rm -it golang:1.19 tail -f /dev/null
docker exec -w /grype -it grype-test bash -c "go install"
docker exec -it grype-test bash -c 'echo "nameserver 8.8.8.9" > /etc/resolv.conf'
docker exec -w /grype -it grype-test bash -c "go run main.go db update"

the output is

 ✔ Vulnerability DB        [checking for update]

No vulnerability database update available
1 error occurred:
        * unable to update vulnerability database: unable to update vulnerability database: unable to download listing: Get "https://toolbox-data.anchore.io/grype/databases/listing.json": dial tcp: lookup toolbox-data.anchore.io on 8.8.8.9:53: read udp 172.17.0.2:39128->8.8.8.9:53: i/o timeout

exit status 1

with exit code 1.

Closes anchore#310 Signed-off-by: Shane Dell <[email protected]>

kzantow · 2023-04-20T19:28:01Z

grype/db/curator.go

 		log.Warnf("unable to check for vulnerability database update")
 		log.Debugf("check for vulnerability update failed: %+v", err)
+		return false, fmt.Errorf("unable to update vulnerability database: %w", err)


I think we may want to only return an error if there is no database to use after this check

But wouldn't you still want return an error if unable to update the database? For example if I had an old outdated database locally and tried to update it but it failed to update, say from a dns error like in this example. If the error is not thrown the command would return that No vulnerability database update available which would be misleading. So this could also cause the user to believe their outdated database is up to date as the returned value of No vulnerability database update available, at least to me, makes me think I have the latest database.

@kzantow Or would something like this work?

stage.Current = "no update available" updateAvailable, metadata, updateEntry, err := c.IsUpdateAvailable() if err != nil { log.Warnf("unable to check for vulnerability database update") log.Debugf("check for vulnerability update failed: %+v", err) stage.Current = fmt.Sprintf("unable to update vulnerability database: %w", err) } ... if stage.Current != "no update available" { return false, errors.New(state.Current) } else { return false, nil }

?

sprathod369 · 2023-05-26T06:21:20Z

The "Fail when grype cant check for db update" issue continues to be sporadic and have been occuring across Grype versions. Recently, I'm facing a similar issue when tried using Grype v0.61.1. On my Ubuntu 18x VM it worked smoothly but it does not work on my RHEL 8x VM. All my VMs are on the same network.

Also when exploring Kubeclarity that integrates with Grype, a similar issue blocks me when I try to setup kubeclarity-grype-server. Below are my K8s logs.

root@xxxx:~/KubeClarity# kubectl logs pod/kubeclarity-kubeclarity-grype-server-7d969b865-9lxks --namespace kubeclarity
time="2023-05-25T11:12:48Z" level=fatal msg="Failed to start scanner: failed to load DB: unable to check for vulnerability database update: unable to download listing: Get \"https://toolbox-data.anchore.io/grype/databases/listing.json\": tls: failed to verify certificate: x509: certificate signed by unknown authority" func=main.run file="/build/grype-server/cmd/grype-server/main.go:53"

kzantow · 2023-06-30T16:20:09Z

Hi all -- what if we modified the grype db update command to fail if it fails to download the database? For users where this is an issue, you could use 2 steps:

grype db update
grype ...

Would this be an acceptable solution where grype itself will still be able to function if there is a slightly old db downloaded (it already complains if the db is too old)?

wagoodman · 2023-07-20T20:41:11Z

Agreed with @kzantow we should not have an exit code of 1 in this case -- grype should work offline without error. We have a separate validator to not use stale DBs incase you're offline for too long (which is configurable). The issue is more about ensuring the UI doesn't lie to the user and say "there is no DB update available" when it didn't check. Instead the UI should really be updated to be clear that it was not able to check for an update and gracefully continue processing.

@shanedell did you want to update this PR to account for the UI update? if not, we can also close this PR instead.

willmurphyscode · 2023-09-12T18:08:36Z

Wanted to add that #1463 will make it so that grype db check will exit zero if the DB is known to be current, 100 if the DB is known to be outdated, and 1 if it can't check?

Maybe grype db check && grype ... will do what you need after that PR is merged?

Signed-off-by: Keith Zantow <[email protected]>

…db-update

cmd/grype/cli/options/database.go

Signed-off-by: Keith Zantow <[email protected]>

willmurphyscode · 2024-07-30T18:10:11Z

cmd/grype/cli/commands/db_update.go

@@ -26,7 +26,9 @@ func DBUpdate(app clio.Application) *cobra.Command {
 }

 func runDBUpdate(opts options.Database) error {
-	dbCurator, err := db.NewCurator(opts.ToCuratorConfig())
+	cfg := opts.ToCuratorConfig()
+	cfg.RequireUpdateCheck = true // always consider an update check failure a db update failure


Do we want to move this to PreRunE so that the config is printed with this change in debug logging?

It turns out the PreRunE didn't work. I think, though, that any grype db <command> which happens to run the db update should, by default, consider failure to check for an update to be an update failure, so moved the default configuration to the appropriate place. Note that this does not currently apply to running the normal grype scan, only the grype db operations (and of those, only the grype db update is using the function in question). This also allows users to disable this behavior if they really want to, using an explicitly configured GRYPE_DB_REQUIRE_UPDATE_CHECK=false...

…db-update

Signed-off-by: Keith Zantow <[email protected]>

shanedell changed the title ~~Fail when grype check cant check for db update~~ Fail when grype cant check for db update Apr 20, 2023

Fail when grype cant check for db update

e38fdba

Closes anchore#310 Signed-off-by: Shane Dell <[email protected]>

shanedell force-pushed the fail-on-cant-check-db-update branch from 60a8519 to e38fdba Compare April 20, 2023 18:14

kzantow reviewed Apr 20, 2023

View reviewed changes

kzantow mentioned this pull request Aug 3, 2023

better errors needed when database can't get updated #265

Open

kzantow added 3 commits July 25, 2024 22:56

chore: make update check failures also fail the scan by default

8eb6837

Signed-off-by: Keith Zantow <[email protected]>

Merge remote-tracking branch 'upstream/main' into fail-on-cant-check-…

43397e3

…db-update

Merge branch 'fail-db-checks' into fail-on-cant-check-db-update

0dc078b

kzantow reviewed Jul 29, 2024

View reviewed changes

cmd/grype/cli/options/database.go Outdated Show resolved Hide resolved

chore: default require update check to false when running grype

0831851

Signed-off-by: Keith Zantow <[email protected]>

willmurphyscode reviewed Jul 30, 2024

View reviewed changes

kzantow added 2 commits July 30, 2024 14:24

Merge remote-tracking branch 'upstream/main' into fail-on-cant-check-…

2d36db8

…db-update

chore: address PR feedback

06b96a0

Signed-off-by: Keith Zantow <[email protected]>

kzantow self-assigned this Jul 31, 2024

wagoodman approved these changes Aug 9, 2024

View reviewed changes

willmurphyscode approved these changes Aug 9, 2024

View reviewed changes

kzantow merged commit d21c549 into anchore:main Aug 15, 2024
10 checks passed

BrewTestBot mentioned this pull request Aug 20, 2024

grype 0.80.0 Homebrew/homebrew-core#181818

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fail when grype cant check for db update #1247

Fail when grype cant check for db update #1247

shanedell commented Apr 20, 2023 •

edited

Loading

kzantow Apr 20, 2023

shanedell Apr 20, 2023 •

edited

Loading

shanedell Apr 21, 2023

sprathod369 commented May 26, 2023

kzantow commented Jun 30, 2023

wagoodman commented Jul 20, 2023

willmurphyscode commented Sep 12, 2023

willmurphyscode Jul 30, 2024

kzantow Jul 30, 2024 •

edited

Loading

Fail when grype cant check for db update #1247

Fail when grype cant check for db update #1247

Conversation

shanedell commented Apr 20, 2023 • edited Loading

kzantow Apr 20, 2023

Choose a reason for hiding this comment

shanedell Apr 20, 2023 • edited Loading

Choose a reason for hiding this comment

shanedell Apr 21, 2023

Choose a reason for hiding this comment

sprathod369 commented May 26, 2023

kzantow commented Jun 30, 2023

wagoodman commented Jul 20, 2023

willmurphyscode commented Sep 12, 2023

willmurphyscode Jul 30, 2024

Choose a reason for hiding this comment

kzantow Jul 30, 2024 • edited Loading

Choose a reason for hiding this comment

shanedell commented Apr 20, 2023 •

edited

Loading

shanedell Apr 20, 2023 •

edited

Loading

kzantow Jul 30, 2024 •

edited

Loading